main modifications

1. sampler
   • always sample 2 * num_beams tokens, even if beam_width < num_beams
2. scheduler
   • update: keep at-most num_beams finished seqs (beam_hyps) and at-most num_beams running seqs in seq_group.
   • move _decode and _stop function to scheduler (from llm_engine).
3. sequence
   • move "data" attributes from Sequence to SequenceData

example

• prompt: "What is deep learning?"
• sampling_params: SamplingParams(temperature = 0, use_beam_search = True, n = 5, max_tokens = 100)
• model & tokenizer: A finetuned Llama
• original beam search returns:

 * Learning is deep type of machine learning that involves using building and neural networks to model and solve complex problems.
 * Deep learning is a subset of artificial intelligence that involves the use of artificial neural networks to model and solve complex problems.
 * It's a type of Artificial allows training artificial neural networks to recognize patterns in data. problems that require
 * the not a word machine learning that involves building and training neural networks to model and generalize knowledge problems that Deep learning is inspired by the structure and function of the human brain and is used for a variety of applications including natural language processing, image and speech recognition, and decision making.
 * A is a the subfield of artificial intelligence that involves the use of artificial neural networks to model and solve complex problems. Deep learning is inspired by the structure and function of the human brain and is used for a variety of applications including natural language processing, image and speech recognition, and decision making.

• modified beam search:

 * Deep learning is a subfield of artificial intelligence that involves the use of artificial neural networks to model and solve problems that require high-level processing, such as natural language processing, image and speech recognition, and decision making.
 * Deep learning is a subset of artificial intelligence that involves the use of artificial neural networks to model and solve complex problems. It is inspired by the structure and function of the human brain and is used for a variety of applications such as natural language processing, image and speech recognition, and decision making.
 * Deep learning is a subfield of artificial intelligence that involves the use of artificial neural networks to model and solve complex problems. Deep learning is inspired by the structure and function of the human brain and is used for a variety of applications including natural language processing, image and speech recognition, and decision making.
 * Deep learning is a subset of artificial intelligence that involves the use of artificial neural networks to model and solve complex problems. It is inspired by the structure and function of the human brain and is used for a variety of applications such as natural language processing, image and speech recognition, and autonomous decision making.
 * Deep learning is a subfield of artificial intelligence that involves the use of artificial neural networks to model and solve problems that require high-level processing, such as natural language processing, image and speech recognition, and decision making. Deep learning is inspired by the structure and function of the human brain and is used to model and solve complex problems.